Bayesian regression based on principal components for high-dimensional data

Motivated by a climate prediction problem, we consider high dimensional Bayesian regression where the number of covariates is much larger than the number of observations. To reduce the dimension of the covariate, the response is regressed on the principal components obtained from the covariates, and it is argued that the PCA regression is equivalent to the original model in terms of prediction. In the PCA regression setting under the sparsity condition, we examine large sample properties of two different modeling strategies: regression with and without covariate selection. For the regression without covariate selection, we obtain the consistency results of the estimators and posteriors with normal priors with constant and decreasing variances, and James-Stein estimator; for the regression with covariate selection, we obtain convergence rates of Bayesian model averaging (BMA) and median probability model (MPM) estimators, and the posterior with variable selection prior. Based on the large sample properties, we conclude that variable selection is essential in high dimensional Bayesian regression. A simulation study also confirms the conclusion. The methodologies are applied to a climate prediction problem. 1 AMS 2000 subject classifications: Primary 62C10; secondary 62J05